The chapter is not a comprehensive collection of all of the ideas included in
the papers on~the~issues. Instead, the focus of this chapter ia to quote or
paraphrase the ideas that illustrate a number of technical data problems which,
when resolved, would improve the quality of the data collection process, when
appropriate, we infuse into the synthesis the relationship of the issues to the
politics of data collection as suggested by the redesign authors.  In chapter H
of this report, we will provide {in our professional judgment) what redirections
NCES should take in its data collection programs to improve the quality of our
Nation's educational data.

METHODOLOGICAL ISSUES
ALTERNATIVE DESIGNS FOR DATA COLLECTION

Sample vs. Universe Surveys. The consensus of the writers who addressed the
"how" of data collection favors the use of representative samples. Universe
surveys came in a distant second.

Cronin's simple and straightforward comment, "I support a simplified sampling
system of educational attainment," reflects the views of a large number of
writers. Cronin states that "data should be drawn from sample rather than total
populations. The information ordinarily will be just as useful and the cost of
data collection, especially to local educators, will be dramatically reduced."
Hill supports this view with his recommendation to reduce the "data reporting
burden by avoiding universal surveys." Harrison supports the use of samples
when appropriate, but suggests the use of administrative record data and
universe surveys when necessary.

The paper by the National Center for Research in Vocational Education calls for
the appropriate use of samples in conducting surveys to provide data for
national policy purposes. Hilliard recommends the selective use of the sample
approach. Implicit within his comments related to surveys of private schools is
the recommendation for sampling the universe.

Samples that provide State-level data are implicit in most recommendations.
Odden and Harrison are explicit in their recommendations about this topic.
Odden states that the sample should be "REPRESENTATIVE FOR EACH OF THE FIFTY
STATES, as well as for the nation as a whole."

Observational Studies. Some writers comment on the collection of some kind of
data by using observational techniques rather than the mail surveys used almost
exclusively by NCES. For example, McPartland, Becker and Grain point out that
"the more that questions deal with behavior patterns determined by 'custom,'
'internal politics,1 or 'general agreement,' and the more that the patterns vary
according to the characteristics of the specific instance, the more that
attention has to be paid to obtaining multiple sources of data about the factor
in question. " They suggest "measuring the behavior or policy at different
points during the school year, and using judgements of external observers rather
than relying solely on self-reports of school practitioners." Peterson suggests
that data pertaining to how teachers allocate their time can be collected by
asking teachers to keep logs, but recommends that data "time spent and student
engaged time must be collected through actual classroom observations." Selden,
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